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ABSTRACT 

Controversy between two opposing factions have 
confused rural development policy formulation. Infrastructure 
advocates believe that the improvement of highways and sewer systems, 
as well as the provision of industrial park sites and industrial 
recruitment, are the keys to successful rural economic growth. Human 
resource development advocates promote structural change in rural 
areas through increased investment: in education and effective 
leadership in rural communities. It cannot be deduced from research 
that infrastructure investment leads directly to regional 
development, yet state and local governments continue to pursue 
industrial recruitment via physical infrastructure development. Lack 
of research on the impact of human resource investment on local or 
regional economic growth causes communities not to value education as 
a means of creating economic opportunities. If the effects of 
increased investment in human resources could be assessed adequately, 
community-devised employment strategies could become a key factor in 
solving the problem of under-utilized labor supplias. In c survey of 
the literature on leadership program evaluation, obstacles that are 
likely to be encountered are presented in this study. This paper 
contains 63 references. (ALL) 
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OBSTACIilS LIMITING THE RESEARCH ON THE ECONOMIC 
IMPACT OF RURAL LEADERSHIP DEVELOPMENT PROGRAMS 

I. INTRODUCTION 

Many scholars (John and Norris, 1989; Smith, 1988; Drabenstotc, et 
al., 1988; Knutson and Fischer) have surmised that the rural economic 
decline of, the 1980s has been largely a product of ineffective or 
nonexistent rural development policy. They have suggested that 
controversy among two factions, in particular, have confused policy 
formulation. 

Traditional policy proponents, or "infrastructure advocates, *• 
believe that improvement of higliways, sewer systems and provision of 
industrial park sites and industrial recruitment are the keys to 
successful rural economic growth. The "newcomers," relatively, to the 
rural development policy platfoirm are the "human resource development 
advocates." This faction has organized initiatives to promote 
structural change in rural areas through increased investments in 
education and effective leadership in rural communities. Although at 
this point they are not recognized as organized lobbyists, supporters of 
human resource investment are perceived by infrastructure advocates as 
threatening to their traditionally accepted role in rural development 
efforts . 

Recent research stresses that areas must be targeted objectively as 
appropriate recipiants of one approach or the other (Ross and Rosenfeld, 
1988; Ryan, 1988; Mulkey and Henry, 1988; DrabensCott and Henry, 1988). 
Thus, understanding the growth potential of an area empirically is 



paramount in gauging more accurately which approach might be most 
effective. Many scholars note, additionally, the lack of theoretical 
underpinnings for defining a concrete relationship between economic 
growth and physical infrastructure development. Attempts to define this 
issue, in fact, have varied considerably in logical approach, scope of 
models, and degree of quantification.^ 

Regional scientists have concluded recently that a variee>' of 
conditions (including when and where) interact with independent 
variables to determine precisely what, if any, economic impact is 
derived from the construction of physical infrastructure such as 
highways ('.sserman, 1939; Nijkamp, 1986). It cannot be deduced from 
research that infrastructure investment will lead a priori to regional 
development improvement. This implies that infrastructure policy is 
only a conditional policy dependent upon a number of regional 
socioeconomic elements. Yet, despite the many unanswered questions and 
uncertainties regarding this field of research, state and local 
governments continue to pursue industrial recruitment via physical 
Infrastructure development strategies (and tax abatements) as a major 
thrust of rural development initiatives. 

Equally puzzling is the complexity of measuring the impact of human 
resource investment on local or regional economic growth. McNamara, 
Xriesel and Deaton (1988) have commenced on the surprising lack of 
research in this area given the significance of human factors to the 



^ See 3riggs. 1980. 1981. 1983; Dodgson. 1973; Fox. 1987; Eyerly. 
et al. 1975; Gaegler, et al., 1979; Gamble, et. al, 1979; Gwilliam, et. 
al, 1966; Hale, 1974; Humphrey and Sell, 1970; Kuehn, 1971; Issennan, e 
al, 1989; Lichter and Fuguitt, 1980; Mohring and Williamson, 1969; 
Nijkamp, 1986; Stephanedes and Eagle. 1986; Twark, et al, 1979). 
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location decision of firms. Smith (1988) has addressed this void in his 
request for evidence to verify the importance or relevance of increased 
investment in education. He argues that economic based impact models, 
used to test assumptions about industrial location in rural areas » are 
conceptually static or short-run in nature. There Is, therefore, no 
provision for analysis of long- term, intergenerational impacts on social 
institutional structure or behavioral predispositions of the indigenous 
population. Smith contends that communities, for lack of contrary 
information, do not place equal value upon education as a means for 
creating economic opportunities. Yet Smith concludes that if the 
effects of increased investment in human resources could be assessed 
adequately, community-devised employment strategies may become a key 
factor in solving the problem of under-utilised labor supplies. 

RURAL LEADER.SHIP Since the 1950s, leadership development programs have 
been used co assist communities in creating goals for localized 
improvements. The focus of these programs has varied from assistance in 
developing growth strategies specifically- to more general education to 
develop public policy awareness. Family Community Leadership Training, 
Public Affairs Leadership Program, Kellogg Farmers' Study Program, 
JCellogg Extension Education Project, and other more recent national, 
state, and local outreach programs have capf.dred the attention of 
legislators, media, and the general public .as a means of educating rural 
residents about public policy. Thus far, these programs have been 
described generally as successful in arousing public awareness or rural 
economic issues and developing interest in directing community 
activities to address these issues (House, 1981). 
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Continuation of stratevide leadership programs outside or in 

addition to privately funded inltia'clves appears to be related to the 

development of ? strong base of support for these initiatives. This 

support depends upon whether the program has been considered 

"economically feasible" (Howell, et al, 1932K But what determines 

economic feasibility? How can we measure whether benefits exceed costs? 

In times of razor-edge budget constraints, it would appear that rural 

development policy choices are greatly contingent upon more quantitative 

verification of the economic value of this aspect of human resource 

investment. Yet, there have been no attempts to measure direct or 

indirect impact on local or regional economic structures. In fact, 

there has been little said about what exactly are the assumed costs and 

benefits of such programs. This problem remains not only unsolved but 

largely untended in academic literature. Is this response a result of 

earlier pitfalls encountered in attempts to test empirically human 

2 

resource development models? Considering plausible reasons explicitly 
as to why this research topic has been "shelved" could provide some 
clues as to whether the question has an empirical answer. Simply 
stated: Can rural leadership programs be evaluated empirically in 
economic terms as are highways? 

In a survey of the literature on leadership program evaluation, 
this paper presents obstacles that are likely to be encountered in 
attempts to address this question. Section i) introduces the issue, 
sections ii) reviews previous research efforts, iii) illustrates 
methodological ard definition obstacles, section iv) reconsiders the 



Mark Blaug (1976) has provided an extensive sur'/ey of 
literature documenting such attempts. 
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research problem, and secCton v) concludes with thoughts regarding 
future efforts. It is hoped, at: least, that the recognition of 
obstacles preventing the economic assessment of such programs will 
spawn innovations to surmount such obstacles. Understanding the 
economic impacts of rural leadership development piro^rams could serve to 
improve these programs. More importantly, however, such information 
could assist citizens and policy-makers in planning more effectively the 
economic survival or revival of rural areas. 

II • PREVIOUS RESEARCH 

Prior to the mid-1970s, limited evaluations of rural leadership 

programs specifically had been conducted. Rothert (1969) used a quasi- 

3 

experimental research design to test for changes in attitudes and 
knowledge among participants in a leadership development program for 
Michigan farmeis. Both pre* and post- tests results indicated that open- 
mindedness and critical Chinking ability were associated with higher 
education levels* The post-tests, however, indicated that for persons 
with lower educational achievement levels the gains were greater, 
Rothert found also increased abilities in reading comprehension skills 
and problem- solving skills regarding farm policy. 

Miller (1976) also evaluated the Michigan Programs based upon the 
participants' reports of their experience. Variables examined included 
changes in participants' lifestyles and self -perception. In addition, 
he used self-reports primarily to examine the extent to which programs 



The type of quasi -experimentation frequently used to evaluate 
lead<>rship programs is the non- equivalent control group method. See 
section iii) for further explanation. 
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influenced an increase in involvement in new community roles, encouraged 
a greater commitment to agricultural activities, family and education, 
and improved communication and decision- making skilli;. 

Giebink (1975) headed an evaluation effo^rc in Montana to assess the 
effects of that program based on self-reports of one group of program 
participants. Similar to Miller's results participants reported 
improvements in their "self-confidence, leadership skills, and knowledge 
of public issues." But several unintended effects, such as familial 
tension and strain, were identified as well. 

Howell and Wilkinson (1977) utilized a non- equivalent control group 
method to evaluate the Pennsylvania program. Pre- and posttest results 
were obtained from program participants at the beginning of the program 
as well as two years after graduation- Their results indicated that 
participants, when compared with a non-equivalent comparison group, had 
statistically significant gains in memberships and officerships in 
economic associations and public-affairs related organizations. "In 
addition, the researchers identified a trend away from participation in 
nongovernment-affiliated voluntary public service organizations and 
toward greater involvement in organizations with legislated authority to 
act on behalf of the community." 

Martin (1977), administering post-tests immediately after the 
program, compared gain scores of the Pennsylvania program participants 
with those of a comparison group. Axso, Martin studied the impact of 
sex, age, and socio-economic status to determine whether different 
effects were experienced by the control groups than for the comparison 
group. Two program groups showed a positive relationship between socio- 
economic status and the political participation variable while one group 
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showed a negative relationship. Age and sex were not found to be 

significantly related to either political participation or memberships 

4 

in instninencal organizations. 

A research team at Washington State University used four separate 
statewide case studies to comparatively evaluate and thus identify 
similarities of program effects (Howell, et al. , 1982).^ Each 
state was analyzed separately using non- equivalent control methods 
likewise. Leadership indicators derived from the program goals measured 
changes in behavior and self-perception. Results were prefaced with 
cautions regarding exogenous factors that threatened validity, however, 
i.e. uncontrolled influences such as variations from state to state in 
type of participants, program objectives, population density and size, 
election processes, and urbanization. As research measurements were 
based on changes regarding involvement in public affairs and expansion 
of extension programs, similarities and/or differences in outcomes were 
used as a basis for overall program effectiveness. Pre-existing 
influential factors such as age, sex, and socioeconomic status, family 
income and pre-test participation on measures of changa in 
organizational structure were accounted for. Although family size and 
motivation was recognized as influential variab7,es, they were excluded 



Martin combined different types of public affairs related 
organizations and economic associations into one measure of 
participation in instrximental organizations, and investigated how the 
program affected persons with different personal characteristics. 

^ The U.K. Kellogg Foundation awarded the contract to evaluate the 
programs and an advisory committee was formed to provide guidelines for 
the research, i.e., define common program goals, research questions and 
evaluation procedures. The four cAse studies evaluated state-wide 
programs in Pennsylvania, Michigan, California, and Montana. 
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from the pre- test indices and researchers advised caution in 
iuterpretirig test results. 

Substantial gains in participation in economic associations were 
noted and were second to government and public service organizations in 
size of average gains from pre- to post- test. Increases were observed 
in involvement in expressive organizations, i.e. , social and fraternity 
groups, as well. Limited data analysis indicated that females increased 
their involvement in public service organizations more so than males who 
participated in programs. Participation priorities and careers were 
affected in terms of competence and advancement toward goals. Some 
strain and tension was noted among families with one spouse in the 
program and among program graduates and peers and older leaders of the 
communitv. Self -assessment indices noted, in particular, an enhanced 
confidence as a public affair participant, as well as increased feelings 
of growth, independence and self -worth. Also, respondents reported 
perceiving themselves as more knowledgeable about resources and 
appreciative of the importance, of information, interrelationships among 
problems, and gains in abilities to use group skills within the setting 
of the community. 

Goal-related program effects overall included new memberships in 
government and public service organizations, increases in appointments 
on regional planning commissions and health councils, and expansion via 
alumni groups to continue education of statewide program participants 
and graduates. Associations were developed during the program and 
spinoff educational programs at involved institutions were created. 

The Montana and Pennsylvania programs received further scrutiny in 
the context of increasing involvement of men and women in public 
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affairs. Cook, et al.,(1985) used the non-equivalent comparison group 
technique to find that both female and male participants in programs 
lasting for more than two years increased their participation in 
instrumental organisations while men showed small Increases in 
participation in expressive organizations. Additionally, there were 
difference.s by sex in changes in particular types of instrumental 
organizations. Men continued to participate more in economic 
organizations than women while women had greater gains than men in 
participation in government organizations. Variations between men and 
women in post- test gains were ascribed to a number of possible 
explanations including program differences, initial minority 
participation by woman in the programs, and prior participation 
differences. 

Williams (1981) evaluated the Montana programs, in particular, 
based on its four variations in duration. Utilizing the "recurrent 
institutional cycle design," ^ self -reports in all groups included 
positive changes in self-image, more effectiveness in leadership 
activities and involvement in roles requiring leadership skills. Those 
participants in more intensive and lengthier program.<i experienced even 
greater gains in leadership indicators, A "self -consciousness scale," 
which measured the extent to which people felt self-conscious when in 
new social situations resulted in gains. 



This type of quasi -experimental research design provides- for 
treatment, i.e., leadership program, to be administered on a cyclical 
schedule to a group of new participants allowing effects of each 
variation on the treatment to be analyzed comparatively. 



III. MEASUREMENT OBSTACLES 



Examination of previous research highlights several apparent pieces 
to the measurement puzzle. First, there is minimal theoretical 
construct on which to base consistent quantitative analysis of 
leadership development programs. Exactly what defines impact cf rural 
leadership development? In fact, no widely accepted or formally 
established definition for leadership exists. Therefore, what 
definition exists for the "impact" in general of rural leader5;hip 
development? The issue is further complicated in that goals in the 
development of leaders are often generally stated and vary considerably 
among programs. It is difficult to determine what are the specific 
program objectives and whether or to what degree they are met. It is 
difficult to discern further what implicit assumption defends any 
economic impact in irural areas as a result of leadership development* 
It appears, however, zhaz programs described as successful are those 
vhose participants display increased activity in economic organizations 
and/or other more "instrumental** groups. Additionally, it appears 
desirable for participants to switch from involvement in expressive 
associations to activity in *?uch groups, and preferably at higher power 
levels. Yet there has been little effort to measure any chaages 
initiated by these organizations as a result of absorbing newly 
developed leaders. 

Methodological obstacles in evaluating any effects of leadership 
development are inherent due to the nature of the development process. 
Rural leadership development programs have been designed around an adult 



education curriculum: a curriculum designed to provide adult citizens 
with information with the intern chat snch information will result in 
desired modified behavior producing and/or nurturing better leaders. 
Research designed to evaluate the impact of such programs has derived 
from the genre of research conducted in education experimentation, i.e., 
reiterative multivariate experimentation with lack of complete contr.>l 
of interactive variables manipulated during the procedures. In fact, m 
most of the research, it is explained at the outset that threats to 
internal and external validity exist ard should be considered in 
interpreting test results. Bu":, it is unclear as to how carefully or 

consistently controls are exercised to ensure greater study validity. 

Part of this problem lies in the fact that the majority of the data 

generated Ls subjective, i.e., to the extent th^t it is based on self- 

reports of "perceived" program impacts. 

DEFINING LEADERSHIP AS AN INDEPENDENT VARIABLE . A considerable body of 
literature, reaching as far back as the 1800s, encompass research 
efforts to define leadership. Most of these earlier attempts focused on 
genetic explanr' I ^ns or the "great man" theory for explaining leader 
qualities. Even as late as the 1900s, energy was spent identifying 
"traits" that comprise desirable characteristics of leaders. Using 
military samples, research then turned to describing leader behavior in 
various roles of authority. The "situational approach" and "contingency 
models," which emphasise identifying leadership characterii'tics demanded 
by groups and specific conditions or situations, grew out of this 
earlier work. Finally, the "transactional approach" developed, which 
examined leadership based on tbe quality of interaction between leaders 
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and followers. 

Research activicics in cooimunity development, iacreasing in the 
1950s, illus*:rate these later trends in understanding leadership « 
Confusion in arriving at a consensus for a definition appears to u^e 
nested in uncertainty as to the kind of person to be a leader and the 
kinds of followers produced. Such connotations included the expert, the 
father -substitute, the natural leader, the manipulator, the community 
organizer, the community educator, and the participant-leader (Biddle, 
1953). 

Rosener (1978) comments additionally on the debate surrounding the 
definition of citizen participation since implementation of The Equal 
Opportunity Act of 1964 that, in Cum, spawned an increase in 
participation and leadership programs. In outlining some standard 
evxluation criteria for which to measure "effective" citizen 
participation, she observes that the problem lies in the fact that 
effective public participation "assumes that there is agreement to its 
meaning, which is not the case." In fact, she xplains, "there is no 
widely acceptable scheme for conceptualizing and measuring its 
effectiveness; and it is, in part, this lack of agreement which prevents 
us from making effective citizen participation the bottom line for 
government. " 

Wengert: (1976) observes that the measurement problem is based in 
par on the assumption that effectiveness is a linear function of 
citizen participation when, in fact, it is curvilinear. Thus 
measurement, he argues, should not assume that more is better and 
instead should seek to me.i:>ure not the maximum but the optimal. . 

To complicate matters farther, goals of more recent prograxns have 
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included a diverse collection of ideas regarding what constitutes the 
development of rural leaders. These include: 

- Increase participation in public affairs activities on the 
part of young men and women from rural areas who show potential 
for leadership . - 

- Improve problem-solving and leadership skills of farmers 
and persons residing in rural areas. - 

- Expand extension programming at land grant universities 
in the areas of public affairs education and rural 
leadership.- (Howell, et al. 1979). 

Leadership development programs with such goals could be graded as 

successful if, in addition, they provide participants with opportunities 

to learn new roles, shaiT)en problem-solving skills, motivate 

participants Co develop new and different roles, broaden their 

comprehension of public issues, provide positive working associations 

with other leaders, and establish new role expectations of peers 

involved in organizatious with program graduates. Program objectives 

have broadened thus in spectrum of desired impact, expanding from 

behavioral changes to include implicit area impacts. 

Another program cited the following goals for leadership 

development:: 

Improve leadership skills of rural leaders, 

enhance rural revitalization, 

address critical issues iu counties, 

develop models and materials for others to use, 

access educational institute knowledge base, 

demonstrate extension's delivery mechanism, and 

cooperate with other agencies and organizations (Palmetto 

Leadership, 1989). 

Such aspirations for developing leaders contrast with numerous 

community development studies that postulate hypotheses for determinants 

of effective leaders as well. "Hierarchical positioning" in the 

community power structure, for example, has been suggested as the 

primary determinant of effective leadership. And, in uhii^ sense, 
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information and access to resources is positively related to position 
in an action network (Beaulieu and Ryan, 1984). According to other 
findings (generally o? supporters of '•mobilization" theory) age, 
education, occupation and other personal characteristics are unimportant 
in determining leadership. Instead, it is organizational involvement 
that breeds later political activity- irrespective of socio-economic 
stacMs. As voluntary associations provide opportunities for development 
of nev relationships its attracts people into public affairs 
organizations --which can nurture leadership skills and develop other 
personal resources as well (Bokemeir and Tait, 1980). Some would argue, 
hovfever, that sex, in particular tends to follow a predictive pattern of 
leadership behavior wich women belonging to expressive organizations and 
men belonging to instrumental. And additionally, chey contend that 
leadership research should address this pattern to comprehend more fully 
the implications that exist for developing effective leaders. 

Perhaps the key to defining laadership lies in understanding 
explicitly vhat skills are necessary to make leaders "effective?" And 
defining effectiveness depends ?n Che conceptual approach the researcher 
takes towards leadership. From a transactional perspective, effective 
leadership may be considered as possessing "the ability to induce 
followers to act on goals that rep^resent the values, wants, and needs of 
both leaders and followers" (Bums, 1^78). Or, leadership may oe thought 
of more as a relationship then as an individual ability: a relationship 
one develops with a potential followership. To be effective, in this 
case "a leader must tailor his or her behavior to fit the specific 
leadership situation." (Vroom, 1973) 

The evaluation literature on rural leadership development, in 
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particular, suggests that development of effective leaders is based on 
improvement of self-image. Implicit in this assumption is that leaders 
who are more self-confident are necessarily more efficient in directing 
a group process to achieve certain ends for the community. This 
rationale appears to regress to earlier assumptions about great men 
being great leaders. Apparently, there is an assumption that rural 
residents possess such low self-esteem that program goals should focus 
on development of self-confidence as a vehicle for bringing about 
organized changes to improve rural areas. Pointing to self-confidence 
as an important factor in "gaining influence," ie., becoming a leader, 
is no;: equivalent necessarily Co developing efficient rural 
organizations with objectives for inducing change --specifically of 
economic variables. 

In contrast, those who support a motivation approach to leadership 
development maintain that people, urban or rural, are more likely to 
accomplish something if they believe they can be effective. The naive 
think they would have to get fifty-one percent to agree first; the 
knowledgeable know that a handful of visible people can sway boards and 
commissions . 

Following the more recently considered transactional view of 

leadership effectiveness, the leader-follower relationship is considered 

in two respects. 

- First, it deals with the responsiveness of the group in 
gaining specified goals. Second, it means securing chose goals 
with the greatest possible consideration for the individuals 
comprising the group.- (Hollander, 1978) 

Leaders are a resource for followers in the transactional view of 

leadership (as opposed to reputation or positional concept of 
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leadership, whereby the leader influences others). In other words the 
transactional leader can allow himself to be influenced by others 
(Hollander, 1978). Research focusing on the follower might be equally 
necessary as it is the follower who accepts or rejects leadership in 
responding to both the leader and the situation. If followers maintain 
expectations of leaders based on contributions and a social exchange 
exists between leaders and followers, does this imply that evaluation of 
leadership performance should concentrate on observational changes in 
followers? If so, what behavior would be desirable from an economic 
perspective? In other words what economic structural changes could be 
measured to trace the effectiveness of developing leaders via the 
followers? 

METHODOLOGICAL OBSTACLES Numerous methodological obstacles exist in the 

evaluation of leadership programs, i.e., whether progrcn; objectives have 

been met and or leaders developed. Given that conformal definition of 

effective leadership existed, the fact remains that attempting to 

measure program effects based upon self-reports and observed behavioral 

changes induces empirical nightmares. 

-In evalxiatioa research, ories are usually based 
upon nonspecific recollections of what was meant to have taken 
place, experiments are often after-the-fact reconstructions of a 
series of events that were supposed to have happened, and the 
measurements are often indirect measures^which may be totally 
unrelated to program activities.- (Howell, et al., 1979) 

In social science research the traditional approach to experimental 

methodology requires random selection of groups, one or more of which 

are subject to a treatment *• or the event under analysis (Campbell and 

Stanley, 1963). Groups not receiving the treatment are analyzed to 

account for changes that occur and are '•controlled*' for exogenous 
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factors that might influence the outcome of the experiment. The basic 
concept is that '•without a control group there is no way to tell how 
much of the overall effect in the experimental group was true cause and 
how much was extraneous effect. "(Bailey 1978) 

Quasi-experimentation, non- equivalent group analysis specifically, 
allows for the absence of this random selection of subjects. For an 
evaluation of behavioral and perceptual changes as a result of 
leadership education, the quasi-experimental techniques are necessary as 
participants are carefully selected based on such criteria as previous 
performance in voluntary organizations, age and educational achievement, 
or socio-economic characteristics. 

Use of this modified experimental methodology, however, is wrought 
with risks to internal and external validity of the experiment. Risks to 
internal validity are those factors that consider "plausible rival 
hypothesis" or those occurrences during the duration of the experiment 
which could make the treatment appear to have had an effect, when in 
fact the effect noted was created by unrelated events. External 
validity concerns itself with interaction effects or those involving the 
treatment and some other variable. Lack of external validity in the 
experimental design prevents generalizations to be made based on 
experimental outcomes. 

■I* 

In evaluating observed and perceived behavioral changes as a result 
of leadership education there exist several pitfalls. In fact, for 
every caution regarding validity in quasi -experimentation there exists a 
threat in evaluation of leadership development. A common problem 
concerns history or those events that occur prior to or betveen . 
treatments or maturation, the "natural process operating within the 
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individual" may induce the observed changes in the treatment group. 
Additionally, "pre- test" experiences might bias "pose- test" scores. 
Instrumentation may pose a validity threat if test instruments are 
altered betveen tests, i.e., type of questionnaire, for example, whether 
the questions change from explicit to generalized. Statistical 
regression is a problem in particular as participants are selected for 
high pre- test scores generally in voluntary behavior : non-equivalent 
selection of control groups may bias post- test results. And, finally, 
mortality or lost cases (i.e., people drop out of programs, move away, 
or die) or those individuals for which only partial data exist can 
introduce sample biases (Campbell and Stanley, 1963) • 

Common risks to external validity of the experiment are: reactive 
testing, whereby pre- test questions affect later behavior; interaction 
effects of selection biases and treatment variations, or when another 
sample produces very different results; the Hawthorne effect, wherein 
participants are aware they are being watched and this in cum affects 
data collected via self-report; the "halo" effect, where respondents 
having invested time in a program see only its merits and want to 
realize its intended affects, thus biasing post- tests; multiple 
treatment interference, or when several treatments administered overlap 
so that differentiation in treatment effects are not possible (Campbell 
and Stanley, 1963; Howell, et al., 1979). External validity problems 
would of course be lessened if control and treatment groups were 
selected randomly yet quasi-experimentatlon precludes this random 
selection. Especially in the case of leadership evaluation, the non- 
equivalent group design does not require pre -experimental sampling 
equivalence between the treatment group and control or comparison group. 
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The longitudinal analysis necessary to assess program effects (for any 
type of human resource investment) emphasizes the weakness of the non- 
equivalent comparative design. In the course of years that comprise a 
given leadership development program, for example, other similar 
treatments may be initiated that try to build upon or imitate 
experimental programs. Complimentary spin-off programs or, in some 
cases, competing programs that may have been administered prior to the 
treatment program if not considered in the group selection process may 
threaten validity with effects of history and multiple treatment 
interference. 

Regional analytic applications of quasi -experimentation to measure 
economic impact of regional policy have attempted to improve validity by 
modifying parametric quasi-experimental techniques utilizing a 
"separate-sample pre-test/post-test control group design." Basically, 
this requires the careful selection of control groups based upon a set 
of determined criteria. Pre-tests are conducted to determine similarity 
between control and treatment groups to ensure that groups selected are 
as similar as possible. The idea being that impact is a measurement of 
change in growth rates assessed over periods of time. 

As regional economic development programs, like leadership 

programs, disallow random selection of study regions, this alternative 

■I* 

has great potential for assessing more accurately economic impact of 
regional policy. And such assessment may include physical 
infrastructure and/or human resource development. Since regional policy 
is designed as treatment for specific regions targeted on the basis of 
need or economic development potential, marriage betveen quasi- . 
experimental techniques and economic impact assessment appears logical. 
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IV. RECONSIDERING THE RESEARCH QUESTION 

Regional development efforts appear to have juxtaposed conceptually 
an emphasis upon education to reduce unemployment and increase socio- 
economic activity (Hansen, 1970) and the development of physical 
infrastructure to provide jobs via industrial development (Fox, 1987). 
Both development policies imply economic impacts would be felt. 
Endorsement of one over the other however is tenable in light of 
contradictory findings of research on highways and inconclusive research 
on human resource development. 

Alternatively, research on a Community Resource Development (CRD) 
input measurement model have proven to be disappointing and somewhat 
confusing to the original motives of CRD of which leadership development 
is an important part. In examining CRD programs in Arkansas in the late 
1950s and 1960s, Miller, et al. (1984), using this method, found that 
programs had no positive effect on emplojonent or economic activity that 
could be measured. In other words job-training increased unemployment 
via increasing the size of labor force and was thought to have increased 
out-migration of young persons as well to other job opportunities. 
Significant "social multipliers," (a commonly assumed basis for CRD 
programs, in that community improvement can occur as a result of local 
activity and interest stimulation which could impact simultaneously 
service and economic development sectors) was not found to have occurred 
in this study. Economic sectors of study counties did not respond 
positively to inputs but those counties with specific targeted problems 
did show positive economic development effects in years following the 
CRD inputs. 

A question arising from this analysis is whether the effective 
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al*:ernative is to focus on examination of goal specific measurement. As 
Miller points out, trying to estimate statistically overall effects 
(quality of life improvements) is inappropriate. Instead, he suggests 
that impact assessment models be constructed that identify carefully 
detailed objectives for area specific analysis and to measure if they 
had been achieved. Furthermore, he questions the use of "community 
development" as an output in terms of evaluation. 

Tne Heartland Center study (1987) has approached the economic 
impact issue by synthesizing what positional and reputational leaders of 
rural communities deemed as attributes that could stimulate growth. 
From among these 20 clues pinpointed as survival necessaries, the 
following allude to intent to concentrate on economic issues. These 
included: emphasis on quality in business and community life, 
willingness to invest in the future, netvorking and resource linking to 
state agencies and programs as apart of an economic development 
approach, knowledge regarding competitive positioning (physical 
infrastructure), a county«wide economic development approach, improving 
quality of life, realistic appraisal of future opportunities, sound and 
well-maintained infrastructure, careful use of fiscal resources, 
sophisticated use of information resources, and a willingness to seek 
help from the outside, yet conviction that, in the long run, you have to 
do "ic" yourself. 

If laader perception is supposed to be a measurable indicator of 
success of leadership development, more attention needs to be paid to 
demographic differences or similarities among leader-participants. 
According to Bachtel and Molnar (1980), some community elements receive 
direct benefits from the attraction of new industry and expansion of 
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business while others do not. Thus, there is an equity and 
redistribution question unanswered and unaddressed. Additionally, 
research has indicated that blacu and white perspectives differ on 
industrial growth issues and reflect different concerns and interests. 
In the future, leadership programs should account for these differences 
in program goal setting, objective targeting, and evaluation procedures. 

IN THE CONTEXT OF INTERVEOTION Napier, et al. (1980) has tested a human 
resource development model to attempt to understand unemployment status 
and duration and investigate attitudes in rural counties toward 
community development priorities. The rationale is that intervention 
programs to reduce unemployment are development approaches based on 
adaptation of human resource development models. Napier has argued that 
the improvement of community resident role playing skills will increase 
opportunities to participate in the economic system. This assumes, 
however, that useful roles are available for persons with completed 
educational experiences. This assumption likewise implies that human 
resource development resource models as de.velopment strategy will be 
more effective in an already diversified economy or geographic regions 
with developed economic infrastructure. There is an obvious need for 
future evaluation of human resource development models in rural areas 
based on the developmental stage of economic structure. Consideration 
for theory regarding the level of physical and social infrastructure in 
an area will have to be incorporated into empirical regional studies as 
a necessary criteria for estimating potential impacts of leadership 
programs . 
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WESTERN RU RAL DEVELOPMENT CE^^^ER An evaluation approach at WRDC (1987) 

is attempting co determine whether economic development can be justified 

as an objective for local leadership groups, especially if initiated 

from the outside.^ The researchers have outlined five steps to 

examine the problem. First, participants who attended a workshop on 

economic development v/ere evaluated using self-reports noting 

behavioral, perceptual and attitude changes as a result of the workshop. 

Second, a follow-up evaluation sheet was administered one year after the 

workshop to observe participants views of changes in their community to 

validate effects of the leadership project, i.e., whether participants 

perceived any changes" related to economic development as a result of the 
8 

program. 

An academic literature search was conducted in hopes of explaining 
income, employment, or any other related changes to explain or offer as 
information to educate citizens about economic potential.^ One hundred 
and thirty citations were identified that demonstrated methods 
most often used to initiate growth* These included: leadership 
training, manpower training, tax incentives, and allocation via 



The following information was volunteered by Russ Youmans via 
telephone interview in July, 1989. 



Contact with participants was maintained but the follow-up 
evaluation was unexpected. 



Any relationships between transportation systems and economic 
growth were included. 
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government funding. 

It was hoped that, as a product of the search, a guide to "what 
works" in other places and situations to stimulate economic growth could 
be derived and presented to agencies and decision makers. Future plans 
are to examine these approaches by sorting through the compilation to 
determine those that intervene in economic development of communities. 
Such examples are expected to include: needs assessments, assessments to 
incite activity, and field representatives to assist citizens in 
accomplishing goals. Three or four of these approaches will be analyzed 
for effectiveness in increasing abilities in communities to deal with 
economic development issues. With these methods in mind a large sample 
of communities will be analyzed in terms of changes in economic 
variables via a cross-sectional analysis. 

V. CONCLUSIONS 

If economic analysts perceive leadership from a positive 
stance so that the basic interest is in how choices and decisions 
"would" be made extending this perception to impact analysis 
warrants identifying what economic impacts "would" occur as a 
result of leadership development. Examining obstacles in defining 
leadership in this context, however, will re<^uire innovative 
methodological considerations to stretch this line of thought. 
Attempts to outline such assumptiom; in the design of the program 
will reduce the ambiguity surrounding the concept of leadership 
development that might prevent public officials and citizens alike 
from being held accountable, A clear consensus of assumptions- 
regarding participation, leadership, and associated effects in a 
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regional perspective will prove useful to those directly and 
indirectly involved in leadership programs. It will require those 
mandating such programs to clarify their expectations and implicit 
goals. Additionally, it will necessitate that citizens be 
informed and knowledgeable about such assumptions in order to make 
reasonable demands. 

In moving toward a method for measuring the economic impact of 
effective leadership, consideration must be given to how well 
organizations absorbing these leaders achieve given available 
information, resources, or limitations. Specific criteria to define 
standards of program evaluation will have to be identified in the design 
phase of programs. Such criteria might include: evidence of formalized 
courses of action and funding. Examples of transacti'^nal leadership 
development, to the extent that citizens have input in policy and 
planning decisions, will have to be observed, as well as that citizens 
were involved in problem definition, alternative evaluation, and 
priority setting. As Rosaner (1978) observes, the value question is do 
activities to meet the criteria serve as end to themselves or as a means 
to an end? And even with an outline of more objective criteria the 
final decision as to "whether the program Is effective is subjeccive but 
at least less so.** 

r 

We may assert with reasonable confidence that economic development 
is unlikely without: leaders. But we cannot say confidently that 
increasing leadership development efforts wil* lead to economic 
development. The question that has evaded scientific inquiry remains 
unanswered. 
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